Image processing apparatus having a plurality of image processing blocks that are capable of real-time processing of an image signal

ABSTRACT

An image processing apparatus is provided which offers higher versatility than conventional image processing apparatuses. When an input signal to a spatial filtering block is a monochrome signal that contains Y component only, a selector selects its input terminal and a selector selects its input terminal. Then, a low-pass filter output signal of a programmable spatial filter is inputted to a spatial filter, and a low-pass filter output signal of the spatial filer is inputted to a spatial filter. That is, the programmable spatial filter and the spatial filters are connected in series (in cascade), and the cascade-connected three spatial filters perform filtering operation. In this example, low-pass filters with 5H5 taps are connected in cascade in three stages, which enables low-pass filtering with 13H13 taps.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. application Ser.No. 13/091,838, filed Apr. 21, 2011, which is a divisional applicationof U.S. application Ser. No. 11/733,958, filed Apr. 11, 2007, which isbased on Japanese Patent Application No. 2006-118018, filed Apr. 21,2006, the entire contents of each of which are incorporated hereinto byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to image processing apparatuses, andparticularly to an image processing apparatus having a plurality ofimage processing blocks that are capable of real-time processing of animage signal inputted from outside.

2. Description of the Background Art

For an image input apparatus such as a digital still camera, JapanesePatent Application Laid-Open No. 2000-236473 discloses a conventionalimage processing apparatus including an imaging device, an analog signalprocessing circuit connected to the imaging device, and a Real-timeProcessing Unit (RPU) connected to the analog signal processing circuit,where the RPU is capable of processing in a real-time manner an imagesignal outputted from the analog signal processing circuit.

However, this conventional image processing apparatus does not offersufficient versatility in the following points:

the number of taps of spatial filter in the spatial filtering block isfixed;

it is not possible to obtain a high-frequency component when the spatialfilter is set not as a high-pass filter;

it is not possible to obtain a mixture signal, such as a signalcontaining center pixel signal and low-frequency component at a desiredratio, as an output signal from the spatial filter;

image quality is considerably deteriorated when a divergence occursbetween a pixel as a source of generation of feature data for modulationand a pixel as a target of modulation of luminance signal;

image quality is considerably deteriorated when a divergence occursbetween a pixel as a source of generation of feature data for modulationand a pixel as a target of modulation of color signal;

no means is provided which is capable of applying high-speed colortransformation to 2-channel color signals (Cb, Cr) outputted from thecolor-space transformation block;

signals are processed as YCbCr color-space signals after the output ofthe color-space transformation block, and only a signal of the samecolor space can be obtained as the final output signal; and

signals are processed as 8-bit-system signals after the output of thegamma correction block, and only a signal of the same number of bits canbe obtained as the final output signal.

SUMMARY OF THE INVENTION

An object of the present invention is to obtain an image processingapparatus having higher versatility than conventional image processingapparatuses.

According to a first aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include a spatialfiltering block that applies a spatial filtering to an input signal. Thespatial filtering block includes a plurality of spatial filters. Whenthe input signal is a multicolor signal, the plurality of spatialfilters perform the spatial filtering in parallel for individualchannels corresponding to individual colors, and when the input signalis a monochrome signal, the plurality of spatial filters are connectedin series to perform the spatial filtering.

According to this configuration, when the input signal is a monochromesignal, the plurality of spatial filters, connected in series, performspatial filtering operation. Thus, an extra spatial filter that isoriginally not related to the processing of monochrome signals can beutilized to achieve filtering operation using spatial filter with alarger number of taps. Also, the total processing time can be shorterthan when a filtering operation using the same spatial filter isrepeated multiple times.

According to a second aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include a pixelinterpolation block that applies a pixel interpolation to an inputsignal. The pixel interpolation block includes a pixel register group.When the input signal to the pixel interpolation block is a signal thatrequires the pixel interpolation, the interpolation is performed, andwhen the input signal to the pixel interpolation block is a signal thatdoes not require the pixel interpolation, a spatial filtering isperformed by a first spatial filter using the pixel register group.

According to this configuration, when the input signal to the pixelinterpolation block is a signal that does not require pixelinterpolation, a spatial filtering using the pixel resister group isperformed. It is thus possible to realize filtering operation usingspatial filter with a larger number of taps. Also, the total processingtime can be shorter. Furthermore, since the first spatial filterperforms spatial filtering using the pixel register group of the pixelinterpolation block, the circuit scale can be smaller than when asimilar pixel register group is newly provided in the spatial filteringblock.

According to a third aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include a spatialfiltering block that applies a spatial filtering to an input signal. Thespatial filtering block includes a first low-pass filter, a programmablespatial filter, and a first operator. The first low-pass filter performsa first low-pass filtering. The first operator subtracts a firstlow-pass filter output signal that is an output signal of the firstlow-pass filter from a center pixel signal that is an original pixelsignal, so as to generate a high-frequency component. The spatialfiltering block is capable of simultaneously outputting at least two ofthe first low-pass filter output signal, an output signal of theprogrammable spatial filter, and the high-frequency component.

According to this configuration, the first operator subtracts the firstlow-pass filter output signal from the center pixel signal to generate ahigh-frequency component. It is thus possible to obtain a high-frequencycomponent similar to a high-pass filter output signal, without using ahigh-pass filter.

According to a fourth aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include a spatialfiltering block that applies a spatial filtering to an input signal. Thespatial filtering block includes a first low-pass filter and a firstoperator. The first low-pass filter performs a first low-pass filtering.The first operator multiplies together a center pixel signal that is anoriginal pixel signal and a first coefficient, multiplies together afirst low-pass filter output signal that is an output signal of thefirst low-pass filter and a second coefficient, and adds the results ofthese multiplications together. The sum of the first coefficient and thesecond coefficient is constant.

According to this configuration, the first operator multiplies togetherthe center pixel signal and a first coefficient, multiplies together thefirst low-pass filter output signal and a second coefficient, and addsthe results of the multiplications together. It is thus possible, bysetting the first and second coefficients to desired values, to obtainan output signal in which the center pixel signal and the first low-passfilter output signal are mixed at a desired ratio. The output signalthus obtained can be arbitrarily utilized according to the purpose. Itis also possible to keep the direct-current component gain at a constantvalue because the sum of the first coefficient and the secondcoefficient is constant.

According to a fifth aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include a spatialfiltering block and a signal modulation block. The spatial filteringblock applies a spatial filtering to an input signal. The signalmodulation block applies a modulation to an output signal from thespatial filtering block as a signal to be modulated. The spatialfiltering block includes a spatial filter that outputs a luminancesignal that has been subjected to the spatial filtering. The signalmodulation block includes a low-pass filter and an operator. Thelow-pass filter applies a low-pass filtering to a pixel unit featuresignal corresponding to a characteristic of each pixel. The operatormodulates the luminance signal by using the pixel unit feature signalthat has been subjected to the low-pass filtering.

According to this configuration, the luminance signal is modulated witha pixel unit feature signal that has been low-pass-filtered. Applyingthe low-pass filtering to the pixel unit feature signal suppressesvariations among pixels. This avoids significant variations in thedegree of modulation of the luminance signal among pixels. As a result,even if a divergence occurs between a pixel as a source of generation ofthe pixel unit feature signal and a pixel as a target of the luminancesignal modulation, it is possible to lower the degree of image qualitydeterioration due to divergence in luminance from proper value, thanwhen the pixel unit feature signal is not low-pass-filtered.

According to a sixth aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include a spatialfiltering block and a signal modulation block. The spatial filteringblock applies a spatial filtering to an input signal. The signalmodulation block applies a modulation to an output signal from thespatial filtering block as a signal to be modulated. The spatialfiltering block includes a spatial filter that outputs a color signalthat has been subjected to the spatial filtering. The signal modulationblock includes a low-pass filter and an operator. The low-pass filterapplies a low-pass filtering to a pixel unit feature signalcorresponding to a characteristic of each pixel. The operator modulatesthe color signal by using the pixel unit feature signal that has beensubjected to the low-pass filtering.

According to this configuration, the color signal is modulated with apixel unit feature signal that has been low-pass-filtered. Applyinglow-pass filtering to the pixel unit feature signal suppressesvariations among pixels. This avoids significant variations in thedegree of modulation of the color signal among pixels. As a result, evenif a divergence occurs between a pixel as a source of generation of thepixel unit feature signal and a pixel as a target of color signalmodulation, it is possible to lower the degree of image qualitydeterioration due to divergence in color from proper value, than whenthe pixel unit feature signal is not low-pass-filtered.

According to a seventh aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include acolor-space transformation block that applies a color-spacetransformation to an input signal. The color-space transformation blockincludes a color-space transformation circuit and a two-dimensionallookup table. The color-space transformation circuit performs thetransformation to output 2-channel color signals. The two-dimensionallookup table follows the color-space transformation circuit andtransforms the 2-channel color signals on the basis of a plurality oftransform values previously defined which represent a correspondencebetween input data value pairs and output data value pairs.

According to this configuration, the two-dimensional lookup table isconnected to follow the color-space transformation circuit that outputstwo-channel color signals. The color-space transformation block performscolor-space transformation by real-time processing. When the color-spacetransformation is performed using a three-dimensional lookup table thatinvolves large amounts of calculations, the processing in thethree-dimensional lookup table forms a bottleneck to delay the entireprocessing. Also, a three-dimensional lookup table involves a very largecircuit scale. In contrast, using a two-dimensional lookup table avoidsincreased circuit scale and achieves high-speed color-spacetransformation without delaying the entire processing.

According to an eighth aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include a firstcolor-space transformation block, a processing block, and a secondcolor-space transformation block. The first color-space transformationblock transforms a signal of a first color space to a signal of a secondcolor space including a luminance component and a plurality of colorcomponents. The processing block applies predetermined image processingto the signal of the second color space outputted from the firstcolor-space transformation block. The second color-space transformationblock transforms the signal of the second color space outputted from theprocessing block into a signal of a color space that is different fromthe second color space.

According to this configuration, the second color-space transformationblock transforms a signal of the second color space outputted from theprocessing block into a signal of a color space that is different fromthe second color space. Thus, the processing block performs variousoperations using signals of the second color space, and then the secondcolor-space transformation block transforms the color space. It is thuspossible to finally obtain an output signal of the first color space,for example.

According to a ninth aspect of the present invention, an imageprocessing apparatus includes a plurality of image processing blocksthat are capable of real-time processing of an image signal inputtedfrom outside. The plurality of image processing blocks include aresizing block that applies an image resizing to an input signal. Theresizing block includes a first resizing circuit and a second resizingcircuit. The first resizing circuit is capable of processing a signalwith a first number of bits. The second resizing circuit is capable ofprocessing a signal with a second number of bits that is smaller thanthe first number of bits. When the input signal is of the first numberof bits, the resizing block performs the resizing by using the firstresizing circuit, and when the input signal is of the second number ofbits, the resizing block performs the resizing by using the secondresizing circuit.

According to this configuration, 12-bit-system signals for main imagesand 8-bit-system signals for display or for thumbnail images, forexample, are inputted as input signals to the resizing block. Then, itis possible to select which resizing circuit should be used according tothe number of bits of the input signal, from among a plurality ofresizing circuits configured to process signals of different numbers ofbits. Furthermore, the circuit scale can be smaller than when theplurality of resizing circuits are all provided as resizing circuitscapable of processing signals of the first number of bits.

These and other objects, features, aspects and advantages of the presentinvention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram schematically illustrating the entireconfiguration of a digital still camera according to a preferredembodiment of the present invention;

FIG. 2 is a block diagram schematically illustrating the functionalconfiguration of an RPU;

FIGS. 3 to 11 are circuit diagrams illustrating a specific configurationof the RPU;

FIG. 12 is a diagram showing an example of setting of transform valuesin a two-dimensional lookup table;

FIG. 13 is a diagram showing another example of the two-dimensionallookup table;

FIGS. 14 and 15 are circuit diagrams showing a first example of specificconfiguration of the two-dimensional lookup table; and

FIGS. 16 and 17 are circuit diagrams showing a second example ofspecific configuration of the two-dimensional lookup table.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A preferred embodiment of the present invention will now be described indetail referring to the drawings. In the drawings, the same referencecharacters indicate the same or corresponding components.

FIG. 1 is a block diagram schematically illustrating the entireconfiguration of a digital still camera according to a preferredembodiment of the present invention. A CCD 1 is followed by an analogsignal processing circuit 2. The analog signal processing circuit 2 isfollowed by an SPU (Sensor Processing Unit) 3. The SPU 3 is followed byan RPU (Real-time Processing Unit) 4. The RPU 4 is connected to a mainbus 5. As well as the RPU 4, a CPU 6, an LCD driver 8, a DMA controller9, a memory interface 10, and a card controller 12 are also connected tothe main bus 5. An LCD 7 is connected to the LCD driver 8, an SDRAM 11is connected to the memory interface 10, and a memory card 13 isconnected to the card controller 12.

The CCD 1 has a three-color-system color filter for, e.g., R (Red), G(Green) and B (Blue), or a four-color-system color filter for, e.g., Y(yellow), M (Magenta), C (Cyan), and W (White), and it pictures an imagewith an optical lens and outputs the image signal thus obtained. Othertype of imaging device, such as a CMOS image sensor, may be used inplace of the CCD 1.

The analog signal processing circuit 2 receives the analog image signalfrom the CCD 1, and applies signal processings to the analog imagesignal, such as noise reduction, signal amplification, A/D conversion,etc., so as to output a digital image signal.

The SPU 3 receives the image signal from the analog signal processingcircuit 2, and applies thereto various signal processings related toproblems due to image sensor characteristics, such as white balancecontrol.

The RPU 4 receives the image signal from the SPU 3, or an image signalfrom the SDRAM 11 through the main bus 5, and applies various signalprocessings thereto, such as pixel interpolation, color-spacetransformation, false color suppression, etc. (which will be fullydescribed later).

The CPU 6 generally controls the operations of the components shown inFIG. 1. However, the transmission of image signals through the main bus5 is controlled by the DMA controller 9, whereby the load on the CPU 6is reduced.

The LCD 7 functions as a finder of the digital still camera. The LCD 7displays a low-resolution image represented by, e.g., an 8-bit-systemimage signal. The term “8-bit system” means eight bits with no sign ornine bits with a sign. Hereinafter, “n-bit system” means n bits with nosign or n+1 bits with a sign. The display of images in the LCD 7 iscontrolled by the LCD driver 8.

When a user presses an image-taking button (not shown), ahigh-resolution image (main image) represented by, e.g., a 12-bit-systemimage signal, is recorded in the memory card 13. The resolution of themain image can be selected by the user. The recording of images in thememory card 13 is controlled by the card controller 12.

FIG. 2 is a block diagram schematically illustrating the functionalconfiguration of the RPU 4. The RPU 4 includes a single pixel processingblock 20, a pixel interpolation block 21, a first gamma-correction block22, a first color-space transformation block 23, a spatial filteringblock 24, a coring block 25, a color suppression block (signalmodulation block) 26, a second color-space transformation block 27, asecond gamma-correction block 28, and a resizing block 29.

The RPU 4 is capable of processing an image signal directly inputtedfrom the SPU 3 (real-time processing) and an image signal once stored inthe SDRAM 11 from the SPU 3 and then transferred from the SDRAM 11through the main bus 5 (post processing).

In real-time processing, all processing blocks 20 to 29 operate on thebasis of a given pixel clock synchronized with a read clock for readingpixel signals from the CCD 1. That is, the processing blocks 20 to 29receive input image signals from the preceding processing blocksaccording to the pixel clock, and process the input image signalsaccording to the pixel clock. All processing blocks 20 to 29 are capableof operating in parallel according to the pixel clock, and so the RPU 4is constructed as a pipeline image processing device. In postprocessing, a pixel clock for the RPU 4 may be set independently of theread clock for the CCD 1.

An image signal inputted to the single pixel processing block 20 fromthe SPU 3 or the SDRAM 11 is sequentially processed in the processingblocks 20 to 29, and then transferred from the resizing block 29 to theSDRAM 11 through the main bus 5. It is also possible to bypass arbitraryone or ones of the processing blocks 20 to 29.

The single pixel processing block 20, the pixel interpolation block 21,and the first gamma-correction block 22 have inputs and outputsconnected to the main bus 5. Accordingly, it is possible to input animage signal to each of the processing blocks 20 to 22 from the SDRAM 11through the main bus 5, and to transfer the image signal, processed inthe processing blocks 20 to 22, to the SDRAM 11 through the main bus 5.For example, it is possible to input an image signal from the SDRAM 11to the first gamma-correction block 22 and transfer the image signal,processed in the first gamma-correction block 22, to the SDRAM 11 fromthe first gamma-correction block 22.

FIGS. 3 to 11 are circuit diagrams illustrating a specific configurationof the RPU 4. These diagrams are connected together at the connectionpoints N₀₀₁ to N₀₇₀ Of the same numbers. Now, referring to FIGS. 3 to11, the specific configurations of the processing blocks 20 to 29 shownin FIG. 2, particularly characteristic portions related to the presentinvention, will be sequentially described below.

The RPU 4 has 4-channel input DMA channels IN-CH0 to IN-CH3 (see FIG. 3,for example), and 4-channel output DMA channels OUT-CH0 to OUT-CH3 (seeFIG. 11, for example). Each DMA channel has 4-channel, color channels C0to C3.

In the notation of the numbers of bits of image signals, “S” and “U”preceding the numbers of bits represent “with a sign” and “with nosign”, respectively. For example, “S17” means 17 bits with a sign, and“U8” means 8 bits with no sign. Also, in the diagrams, a reduction ofthe number of bits indicates “clipping” of omitting low-order bits aftera multiplier, or omitting high-order bits after an adder or a shifter,unless otherwise stated.

Single Pixel Processing Block 20

The configuration of the single pixel processing block 20 is illustratedin FIG. 3. As shown in FIG. 3, the single pixel processing block 20includes selectors 101 to 109, multipliers 110 to 114, adders 115 and116, shifters/limiters 117, 118, and a divider 119. The referencecharacters A1, A2, B1, B2 are coefficients, E0 to E3 are offsetcoefficients for individual color channels C0 to C3, OB0 to OB3 arecoefficients for optical black correction for individual DMA channelsCH0 to CH3, and WB0 to WB3 are coefficients for white balance correctionfor individual DMA channels CH0 to CH3.

The single pixel processing block 20 has a function of processing, pixelby pixel, an image signal (Input Data) 120 inputted from the SPU 3, andan image signal inputted from the SDRAM 11 through the main bus 5, andit is capable of selectively performing temporal averaging, shadingcorrection, etc.

The temporal averaging is a process of averaging an image signalinputted to the RPU 4 over a plurality of frames or a plurality offields, while reading pixel signals from the CCD 1 according to the readclock. The scheme of addition of pixel signals for averaging can beselected from cumulative addition and circulating addition, according tothe settings of the selectors 107 and 108 and the coefficients B1 andB2. Cumulative addition is performed when the selector 107 selects itsinput terminal 1072, the selector 108 selects its input terminal 1082,and the coefficients B1 and B2 are both set to “1”. On the other hand,circulating addition is performed when the selector 107 selects itsinput terminal 107 ₁, the selector 108 selects its input terminal 1082,and the coefficient B2 is set to “∀”.

Also, the shading correction can be performed when the selector 107selects its input terminal 107 ₂, the selector 108 selects its inputterminal 1081, and the coefficient B1 is set to “0”. In this case, agiven shading correction parameter is inputted to the input terminal 108₁ of the selector 108 from the SDRAM 11 through the main bus 5 and theselector 106.

Pixel Interpolation Block 21

The configuration of the pixel interpolation block 21 is illustrated inFIGS. 4 and 5. As shown in FIGS. 4 and 5, the pixel interpolation block21 includes a pixel register group 201, line buffers (FIFOs) 202 to 207,and an interpolation operation unit 208. The pixel register group 201 isformed of a total of 49 pixel registers of 7H7 taps around a pixelregister 201 c in which the pixel signal of a target pixel is stored.The line buffers 202 to 207 are connected between the rows of the pixelregister group 201. For example, the line buffer 202 is connectedbetween the first and second rows of the pixel register group 201.

The interpolation operation unit 208 receives the 49 pixel signalsstored in the pixel registers of the pixel register group 201. On thebasis of the input pixel signals, the interpolation operation unit 208interpolates a color component absent for the target pixel, by referringto the pixel signals around the target pixel. For example, when thetarget pixel is a pixel having R component in the RGB color space, theabsent G and B components are generated by interpolation. Then, thecolor channel C0 output terminal of the interpolation operation unit 208outputs R-component pixel signal, the color channel C1 output terminaloutputs G-component pixel signal, and the color channel C2 outputterminal outputs B-component pixel signal. The color channel C3 outputterminal outputs, as needed, a KEY signal for particularly featuringeach pixel. When the input signal to the RPU 4 is a 4-color-systemsignal of Y, M, C, W, then the color channel C0 output terminal outputsY-component pixel signal, the color channel C1 output terminal outputsM-component pixel signal, the color channel C2 output terminal outputsC-component pixel signal, and the color channel C3 output terminaloutputs W-component pixel signal.

The interpolation operation unit 208 also has a function of determiningthe degree of correlation between target and surrounding pixels using anarbitrary correlation determining scheme (e.g., vertical/horizontalcorrelation determination) on the basis of the pixel signals inputtedfrom the pixel register group 201, and the resultant correlation signalis outputted from the channel K0 (channel C4) or channel K1 (channel C5)output terminal.

The interpolation operation unit 208 also has a function of detectingcolor level (chroma, saturation) of a target pixel on the basis of thepixel signals inputted from the pixel register group 201, and theresultant color level signal is outputted from the channel K0 or channelK1 output terminal.

Also, the interpolation operation unit 208 has a function of performingSobel filtering for edge detection on the basis of the pixel signalsinputted from the pixel register group 201, and the resultant Sobelfilter output signal is outputted from the channel K0 or channel K1output terminal.

These correlation signal, color level signal, and Sobel filter outputsignal represent features of each pixel, and they can be regarded as“pixel unit feature signals” corresponding to characteristics of eachpixel. It is possible to arbitrarily select which of the correlationsignal, color level signal, and Sobel filter output signal should beoutputted from the channels K0 and K1. It is also possible to output oneof the pixel unit feature signals as the above-mentioned KEY signal fromthe color channel C3 output terminal.

First Gamma-Correction Block 22

The configuration of the first gamma-correction block 22 is illustratingin FIG. 5. As shown in FIG. 5, the first gamma-correction block 22includes a gamma correction unit 301. The gamma correction unit 301 iscapable of performing conversion from 16-bit system to 16-bit systemusing, e.g. lookup tables, independently for individual color channelsC0 to C3.

The gamma correction unit 301 is preceded by a linear matrix converter302. The linear matrix converter 302 performs color corrections inlinear region, such as correction of difference from idealcharacteristics of the color filter of the CCD 1.

When selectors 209 to 212 respectively select their input terminals 209₂ to 212 ₂ and selectors 303 to 306 respectively select their inputterminals 303 ₂ to 306 ₂, the output signals from the interpolationoperation unit 208 can be inputted to the gamma correction unit 301.When the selectors 303 to 306 respectively select their input terminals303 ₁ to 306 ₁ and a selector 308 selects its input terminal 308 ₁, anoutput signal from the SPU 3 can be inputted to the gamma correctionunit 301. When the selectors 303 to 306 respectively select their inputterminals 303 ₁ to 306 ₁ and the selector 308 selects its input terminal308 ₂, an output signal from the shifter/limiter 118 of the single pixelprocessing block 20 can be inputted to the gamma correction unit 301.Also, when the selectors 303 to 306 respectively select their inputterminals 303 ₁ to 306 ₁ and the selector 308 selects its input terminal308 ₃, an image signal read from the SDRAM 11 can be inputted to thegamma correction unit 301 through a color sampling module 307.

The output signals from the gamma correction unit 301 are inputted to aclipping circuit 310 through a selector 309, clipped in the clippingcircuit 310 to 12-bit-system image signals, and then inputted to thefollowing first color-space transformation block 23. The output signalsfrom the gamma correction unit 301 can also be inputted to the SDRAM 11through the selector 309 and the main bus 5. Pixel unit feature signalsoutputted from the channels K0, K1 of the interpolation operation unit208 are inputted to the following first color-space transformation block23 through the selector 309 and the clipping circuit 310.

First Color-Space Transformation Block 23

The configuration of the first color-space transformation block 23 isillustrated in FIG. 6. As shown in FIG. 6, the first color-spacetransformation block 23 includes a color-space transformation circuit401 and a two-dimensional lookup table 402.

The color-space transformation circuit 401 is capable of performingmatrix operation with 4-channel inputs and 4-channel outputs, which, forexample, transforms an RGB color-space or YMCW color-space image signalinto the YCbCr color space (YUV color space) and outputs the imagesignal. The color channel C0 output terminal of the color-spacetransformation circuit 401 outputs Y-component pixel signal, the colorchannel C1 output terminal outputs Cb-component pixel signal, and thecolor channel C2 output terminal outputs Cr-component pixel signal. Thecolor channel C3 output terminal outputs a KEY signal as needed.

The two-dimensional lookup table 402 is connected to the outputterminals of the color channels C1 and C2 of the color-spacetransformation circuit 401, and it transforms the values of theCb-component and Cr-component pixel signals to desired values, on thebasis of a plurality of transform values representing a correspondencebetween pairs of input data values (Cb, Cr) and pairs of output datavalues (Cb, Cr). The two-dimensional lookup table 402 will be describedin detail later.

An exposure determining evaluator 403 is connected to the color channelC0 output terminal of the color-space transformation circuit 401. Forpreconditions to determine shutter speed and diaphragm stop, theexposure determining evaluator 403 determines exposure level on thebasis of proper luminance of the actual image signal. It divides oneframe of image into a plurality of blocks and performs a luminanceevaluation to level the luminances of the blocks.

Spatial Filtering Block 24

The configuration of the spatial filtering block 24 is illustrated inFIGS. 4, 7 and 8. As shown in FIG. 4, the spatial filtering block 24includes a 3H3-tap fixed low-pass filter 501 (LPF-L), a 5H5-tap fixedlow-pass filter 502 (LPF-LL), a 7H7-tap fixed low-pass filter 503(LPF-LLL), and, as shown in FIG. 7, a 5H5-tap programmable spatialfilter 504 corresponding to the color channel C0, a 5H5-tap spatialfilter 505 corresponding to the color channel C1, and a 5H5-tap spatialfilter 506 corresponding to the color channel C2. While FIG. 4 does notshow the settings of filter coefficients of the fixed low-pass filters501 to 503, the total value of filter coefficients is “16” for the fixedlow-pass filter 501, “64” for the fixed low-pass filter 502, and “128”for the fixed low-pass filter 503.

Referring to FIG. 4, the fixed low-pass filters 501 to 503 performlow-pass filtering on the basis of the pixel signals inputted from thepixel register group 201. Referring to FIGS. 4 and 5, when the selectors209 to 212 respectively select their input terminals 209 ₁ to 212 ₁ andthe selectors 303 to 306 respectively select their input terminals 303 ₂to 306 ₂, the center pixel signal as an original pixel signal stored inthe pixel register 201 c and the output signals from the fixed low-passfilters 501 to 503 are inputted to the gamma correction unit 301. Thesesignals outputted from the gamma correction unit 301 are clipped in theclipping circuit 310 to 12-bit-system signals, and they can be inputtedto the programmable spatial filter 504 and the spatial filters 505 and506 shown in FIG. 7 through the color-space transformation circuit 401and the two-dimensional lookup table 402 shown in FIG. 6.

Referring to FIG. 7, the programmable spatial filter 504 is capable ofoutputting a center pixel signal (center), a programmable filter outputsignal (Prog.), a low-pass filter output signal (LPF-L) from 3H3-tapfixed low-pass filter, a low-pass filter output signal (LPF-LL) from5H5-tap fixed low-pass filter, a 3H3-tap Sobel filter output signal(Sobel-33), and a 5H5-tap Sobel filter output signal (Sobel-55). Theprogrammable spatial filter 504 is normally set as high-pass filter, andthe programmable filter output signal (Prog.) is normally a high-passfilter output signal.

The spatial filters 505 and 506 are each capable of outputting a centerpixel signal (center), a low-pass filter output signal (LPF-L) from3H3-tap fixed low-pass filter, and a low-pass filter output signal(LPF-LL) from 5H5-tap fixed low-pass filter.

When the input signal to the spatial filtering block 24 is a color imagesignal (in this example, a YCbCr color-space signal), a selector 508selects its input terminal 508 ₁ and a selector 509 selects its inputterminal 509 ₁. In this case, the Y-component image signal is inputtedto the programmable spatial filter 504, the Cb-component image signal isinputted to the spatial filter 505, and the Cr-component image signal isinputted to the spatial filter 506. Then, the programmable spatialfilter 504 and the spatial filters 505 and 506 perform spatial filteringoperations in parallel.

On the other hand, when the input signal to the spatial filtering block24 is a monochrome image signal including Y component only, the selector508 selects its input terminal 508 ₂ and the selector 509 selects itsinput terminal 509 ₂. Then, the low-pass filter output signal (LPF-LL)of the programmable spatial filter 504 is inputted to the spatial filter505, and the low-pass filter output signal (LPF-LL) of the spatialfilter 505 is inputted to the spatial filter 506. That is, theprogrammable spatial filter 504 and the spatial filters 505 and 506 areconnected in series (in cascade), and the cascade-connected, threespatial filters perform filtering operation. In this example, 5H5-taplow-pass filters are cascade-connected in three stages, enablinglow-pass filtering with 13H13 taps.

In this way, when the input signal is a monochrome signal, the filteringoperation is performed by the cascade-connected multiple spatial filters504 to 506. The spatial filters 505 and 506 that are originally notrelated to the processing of monochrome signals can thus be utilized toenable filtering operation using spatial filter with a larger number oftaps. Also, the processing can be done through a single pass, and so thetotal processing time can be shorter than when the filtering operationusing the spatial filter 504 is repeated through a plurality of passes.Furthermore, because the cascade-connected spatial filters 504 to 506are all low-pass filters, the filter coefficient distribution of thelow-pass filters can be maintained even though a plurality of spatialfilters are cascade-connected, and the function as low-pass filter isnot damaged.

Also, with a monochrome image signal which does not need the pixelinterpolation in the pixel interpolation block 21, the selectors 209 to212 shown in FIG. 5 can respectively select the input terminals 209 ₁ to212 ₁. Accordingly, for example, the output signal from the 5H5-tapfixed low-pass filter 502 shown in FIG. 4 can be inputted to theprogrammable spatial filter 504 shown in FIG. 7. That is, the spatialfilter using the pixel register group 201 shown in FIG. 4 and thespatial filters 504 to 506 shown in FIG. 7 are continuously applied toperform spatial filtering. In this example, 5H5-tap low-pass filters arecascade-connected in four stages, enabling 17H17-tap low-pass filtering.

Thus, when the input signal to the pixel interpolation block 21 does notrequire pixel interpolation, a spatial filtering operation with anincreased number of taps can be realized by using the fixed low-passfilter 502 and the spatial filters 504 to 506. Also, the processing canbe done through a single pass, and so the total processing time can beshorter than when the filtering operation using the spatial filter 504is repeated through a plurality of passes. Furthermore, since the fixedlow-pass filters 501 to 503 perform filtering by using the pixelregister group 201 in the pixel interpolation block 21, the circuitscale can be smaller than when a similar pixel register group is newlyprovided in the spatial filtering block 24. Moreover, because thecascade-connected spatial filters 502, 504 to 506 are all low-passfilters, the filter coefficient distribution of the low-pass filters canbe maintained even though a plurality of spatial filters arecascade-connected, and the function as low-pass filter is not damaged.

Referring to FIG. 7, a noise reduction circuit 507 is connected to theprogrammable spatial filter 504. The noise reduction circuit 507receives the center pixel signal and pixel signals of a pixel registergroup (not shown) from the programmable spatial filter 504. Then, itcompares the maximum and minimum values of the pixel signals of thepixel register group and the value of the center pixel signal, and whenthe value of the center pixel signal coincides with the maximum value orthe minimum value, it recognizes the center pixel signal as noise. Inthis case, a new center pixel signal is generated by interpolation usingsurrounding pixel signals. On the other hand, when the value of thecenter pixel signal does not coincide with the maximum value nor theminimum value, that center pixel signal is recognized not as noise, andno new center pixel signal is generated.

Next, referring to FIG. 8, the spatial filtering block 24 includessubtracters 510 to 512, 519, 520, 523, 524, multipliers 516, 517, 531,532, adders 518, 533, 535, selectors 513 to 515, 521, 522, 525, 526,530, 536, and a limiter 534.

The subtracter 510 subtracts the 3H3-tap low-pass filter output signalfrom the center pixel signal of the programmable spatial filter 504, soas to generate a relatively narrow-band, high-frequency component(HF-N). The subtracter 511 subtracts the 5H5-tap low-pass filter outputsignal from the center pixel signal of the programmable spatial filter504, so as to generate a relatively wide-band, high-frequency component(HF-W). Accordingly, high-frequency components similar to a high-passfilter output signal can be obtained even when the programmable spatialfilter 504 is set not as a high-pass filter.

The subtracter 512 subtracts the 5H5-tap low-pass filter output signalfrom the 3H3-tap low-pass filter output signal of the programmablespatial filter 504, so as to generate a medium-frequency component (MF).It is thus possible to generate a medium-frequency component similar toa band-pass filter output signal without using a band-pass filter thatpasses medium-frequency component. The medium-frequency component thusobtained can be arbitrarily utilized according to the purpose of theuser, whereby the versatility can be enhanced.

The selector 513 selects and outputs one of the high-pass filter outputsignal of the programmable spatial filter 504, the output signal fromthe subtracter 510, and the output signal from the subtracter 511. Theselector 514 selects and outputs one of an output signal from theprogrammable spatial filter 504 (particularly, the output signalprovided when the programmable spatial filter 504 is set as a low-passfilter), the 3H3-tap low-pass filter output signal, and the 5H5-taplow-pass filter output signal.

As shown in FIG. 8, the spatial filtering block 24 simultaneouslyoutputs a plurality of signals, specifically, the center pixel signal ofthe programmable spatial filter 504, an output signal of the selector513 (i.e., the high-pass filter output signal of the programmablespatial filter 504, the high-frequency component outputted from thesubtracter 510, or the high-frequency component outputted from thesubtracter 511), an output signal of the selector 514 (i.e., the 3H3-taplow-pass filter output signal, or the 5H5-tap low-pass filter outputsignal), and the medium-frequency component outputted from thesubtracter 512.

Also, the selector 515 selects and outputs one of the center pixelsignal of the programmable spatial filter 504, the output signal fromthe selector 513, the output signal from the subtracter 512, the outputsignal from the selector 514, the 3H3-tap Sobel filter output signal,and the 5H5-tap Sobel filter output signal.

The multiplier 516 multiplies together the output signal (ahigh-frequency component) from the selector 513 and an arbitrarycoefficient (RATHF) to provide an output, and the multiplier 517multiplies together the output signal (medium-frequency component) fromthe subtracter 512 and an arbitrary coefficient (RATMF) to provide anoutput. The adder 518 adds together the output signal from themultiplier 516 and the output signal from the multiplier 517 to providean output. The high-frequency component and medium-frequency componentcan thus be mixed at a desired ratio, by setting the coefficients(RATHF, RATMF) at desired values. The output signal from the adder 518can be arbitrarily utilized according to the purpose of the user,offering enhanced versatility.

The selector 530 selects and outputs one of a coefficient (RATCNT) and acoefficient (64-∀). The multiplier 531 multiplies together the centerpixel signal of the programmable spatial filter 504 and the outputsignal from the selector 530 to provide an output. The multiplier 532multiplies together the output signal from the selector 514 and a givencoefficient to provide an output. This given coefficient is set to makea constant sum (desirably, “1”) with the coefficient (64-∀) inputted tothe selector 530. The direct-current component gain can be kept at aconstant value because the sum of the coefficients is a constant value.The adder 533 adds together the output signal from the multiplier 531and the output signal from the multiplier 532 to provide an output. Itis possible, by setting the coefficient (64-∀) at a desired value, toobtain an output signal in which the center pixel signal andlow-frequency component are mixed at a desired ratio. This output signalcan be arbitrarily utilized according to the purpose, offering enhancedversatility.

The adder 535 adds together the output signal from the adder 533 and theoutput signal from the limiter 534 to provide an output. The outputsignal from the adder 533 contains direct-current component andlow-frequency component, and the output signal from the limiter 534contains high-frequency component and medium-frequency component.Accordingly, an output signal that contains direct-current component,low-frequency component, medium-frequency component, and high-frequencycomponent is obtained by the adder 535 adding the output signal from theadder 533 and the output signal from the limiter 534. This output signalcan be arbitrarily utilized according to the purpose, offering enhancedversatility.

The selector 536 selects and outputs one of the output signal from theadder 535 and the output signal from the selector 515.

The subtracter 519 subtracts the 3H3-tap low-pass filter output signalfrom the center pixel signal of the spatial filter 505 shown in FIG. 7,so as to generate a relatively narrow-band high-frequency component(HF-N). The subtracter 520 subtracts the 5H5-tap low-pass filter outputsignal from the center pixel signal of the spatial filter 505, so as togenerate a relatively wide-band high-frequency component (HF-W).

The selector 521 selects and outputs one of the 3H3-tap low-pass filteroutput signal of the spatial filter 505 and the 5H5-tap low-pass filteroutput signal. The selector 522 selects and outputs one of the centerpixel signal of the spatial filter 505, the output signal from thesubtracter 519, the output signal from the subtracter 520, and theoutput signal of the selector 521.

The subtracter 523 subtracts the 3H3-tap low-pass filter output signalfrom the center pixel signal of the spatial filter 506 shown in FIG. 7,so as to generate a relatively narrow-band high-frequency component(HF-N). The subtracter 524 subtracts the 5H5-tap low-pass filter outputsignal from the center pixel signal of the spatial filter 506, so as togenerate a relatively wide-band high-frequency component (HF-W).

The selector 525 selects and outputs one of the 3H3-tap low-pass filteroutput signal of the spatial filter 506 and the 5H5-tap low-pass filteroutput signal. The selector 526 selects and outputs one of the centerpixel signal of the spatial filter 506, the output signal from thesubtracter 523, the output signal from the subtracter 524, and theoutput signal of the selector 525.

Coring Block 25

The configuration of the coring block 25 is illustrated in FIG. 8. Asshown in FIG. 8, the coring block 25 includes a coring circuit 601connected to follow the adder 518, a coring circuit 602 connected tofollow the selector 522, and a coring circuit 603 connected to followthe selector 526.

Color Suppression Block 26

The configuration of the color suppression block (signal modulationblock) 26 is illustrated in FIGS. 8 to 10. As shown in FIG. 10, thecolor suppression block 26 includes a spatial filter 739 correspondingto the color channel C3, a spatial filter 753 corresponding to thechannel C4, and a spatial filter 762 corresponding to the channel C5.

The spatial filters 739, 753 and 762 are all 5H5-tap spatial filters,and they are each capable of outputting a center pixel signal (center),a low-pass filter output signal (LPF-L) from 3H3-tap fixed low-passfilter, and a low-pass filter output signal (LPF-LL) from 5H5-tap fixedlow-pass filter.

A selector 736 receives as its input a KEY signal for modulationcorresponding to the color channel C3 (or a fourth-color pixel signal)from the interpolation operation unit 208 shown in FIG. 5. The selector736 is also capable of receiving as its inputs other modulation signalsfrom the SDRAM 11 through the input DMA channels IN-CH2 and IN-CH3. Theselector 736 selects and outputs one of the input signals. The outputsignal from the selector 736 is absolutized in an absolutizing circuit737, and inputted to one input terminal of a selector 738. The otherinput terminal of the selector 738 directly receives the output signalfrom the selector 736. The selector 738 selects and outputs one of theinput signals. The output signal from the selector 738 is inputted tothe spatial filter 739 and low-pass-filtered in the spatial filter 739.

A subtracter 740 subtracts the 3H3-tap low-pass filter output signalfrom the center pixel signal of the spatial filter 739, so as togenerate a relatively narrow-band, high-frequency component (HF-N). Asubtracter 741 subtracts the 5H5-tap low-pass filter output signal fromthe center pixel signal of the spatial filter 739, so as to generate arelatively wide-band, high-frequency component (HF-W).

A selector 742 selects and outputs one of the center pixel signal andthe high-pass filter output signal of the programmable spatial filter504 shown in FIG. 7, the center pixel signal of the spatial filter 739shown in FIG. 10, the output signal from the subtracter 740, the outputsignal from the subtracter 741, and the 3H3-tap low-pass filter outputsignal and the 5H5-tap low-pass filter output signal of the spatialfilter 739.

The output signal from the selector 742 is absolutized in anabsolutizing circuit 743 and inputted to one input terminal of aselector 744. The other input terminal of the selector 744 directlyreceives the output signal from the selector 742. The selector 744selects and outputs one of the input signals.

A selector 750 receives as its input a pixel unit feature signal formodulation corresponding to the channel K0 (channel C4), from theinterpolation operation unit 208 shown in FIG. 5. The selector 750 isalso capable of receiving as its inputs other modulation signals fromthe SDRAM 11 through the input DMA channels IN-CH2 and IN-CH3. Theselector 750 selects and outputs one of the input signals. The outputsignal from the selector 750 is absolutized in an absolutizing circuit751 and inputted to one input terminal of a selector 752. The otherinput terminal of the selector 752 directly receives the output signalfrom the selector 750. The selector 752 selects and outputs one of theinput signals. The output signal from the selector 752 is inputted tothe spatial filter 753 and low-pass-filtered in the spatial filter 753.

In this way, the spatial filter 753 applies low-pass filtering to thepixel unit feature signal corresponding to the channel K0, wherebyvariations among individual pixels can be suppressed. This makes itpossible to avoid considerable variations in the degree of modulation ofindividual pixels, when modulating the Y component (color channel C0),Cb component (color channel C0, and Cr component (color channel C2) onthe basis of the pixel unit feature signal corresponding to the channelK0. As a result, even when a divergence occurs between a pixel as thesource of generation of the pixel unit feature signal and a pixel as thetarget of modulation, it is possible to lower the degree ofdeterioration of image quality due to luminance or color divergence fromthe proper value, than when the pixel unit feature signal is notlow-pass-filtered.

A selector 754 selects and outputs one of the center pixel signal andthe high-pass filter output signal of the programmable spatial filter504 shown in FIG. 7, and the center pixel signal, the 3H3-tap low-passfilter output signal, and the 5H5-tap low-pass filter output signal ofthe spatial filter 753 shown in FIG. 10.

The output signal from the selector 754 is absolutized in anabsolutizing circuit 755 and inputted to one input terminal of aselector 756. The other input terminal of the selector 756 directlyreceives the output signal from the selector 754. The selector 756selects and outputs one of the input signals.

A selector 759 receives as its input a pixel unit feature signal formodulation corresponding to the channel K1 (channel C5) from theinterpolation operation unit 208 shown in FIG. 5. The selector 759 isalso capable of receiving as its inputs other modulation signals fromthe SDRAM 11 through the input DMA channels IN-CH2, IN-CH3. The selector759 selects and outputs one of the input signals. The output signal fromthe selector 759 is absolutized in an absolutizing circuit 760 andinputted to one input terminal of a selector 761. The other inputterminal of the selector 761 directly receives the output signal fromthe selector 759. The selector 761 selects and outputs one of the inputsignals. The output signal from the selector 761 is inputted to thespatial filter 762 and low-pass-filtered in the spatial filter 762.

In this way, the spatial filter 762 applies low-pass filtering to thepixel unit feature signal corresponding to the channel K1, wherebyvariations among individual pixels can be suppressed. This makes itpossible to avoid considerable variations in the degree of modulation ofindividual pixels, when modulating the Y component, Cb component, and Crcomponent on the basis of the pixel unit feature signal corresponding tothe channel K1.

A selector 763 selects and outputs one of the center pixel signal andthe high-pass filter output signal of the programmable spatial filter504 shown in FIG. 7, and the center pixel signal, the 3H3-tap low-passfilter output signal, and the 5H5-tap low-pass filter output signal ofthe spatial filter 762 shown in FIG. 10.

The output signal from the selector 763 is absolutized in anabsolutizing circuit 764 and inputted to one input terminal of aselector 765. The other input terminal of the selector 765 directlyreceives the output signal from the selector 763. The selector 765selects and outputs one of the input signals.

Referring to FIG. 8, a selector 705 receives the center pixel signal ofthe programmable spatial filter 504 shown in FIG. 7, the 3H3-taplow-pass filter output signal, and the 5H5-tap low-pass filter outputsignal. The selector 705 selects and outputs one of the input signals.

Referring to FIG. 10, the output signal from the selector 744 ismodulated in a luminance modulation unit 746 on the basis of theluminance component output signal from the selector 705 shown in FIG. 8,and then further modulated in a lookup table 747. The output signal fromthe selector 756 is modulated in a luminance modulation unit 757 on thebasis of the luminance component output signal from the selector 705shown in FIG. 8, and then further modulated in a lookup table 758. Theoutput signal from the selector 765 is modulated in a luminancemodulation unit 766 on the basis of the luminance component outputsignal from the selector 705 shown in FIG. 8, and then further modulatedin a lookup table 767. A selector 749 selects and outputs one of theoutput signals from the lookup tables 747, 758 and 767.

A selector 745 selects and outputs one of the center pixel signal of thespatial filter 739, the 3H3-tap low-pass filter output signal, and the5H5-tap low-pass filter output signal. The output signal from theselector 745 is modulated by a lookup table 748.

Referring to FIG. 8, a selector 706 selects and outputs one of theoutput signal from the subtracter 512, the high-pass filter outputsignal, the 3H3-tap Sobel filter output signal, and the 5H5-tap Sobelfilter output signal of the programmable spatial filter 504 shown inFIG. 7. The output signal from the selector 706 is absolutized in anabsolutizing circuit 707. The output signal from the absolutizingcircuit 707 is modulated in a luminance modulation unit 718 shown inFIG. 9 on the basis of the luminance component output signal from theselector 705 shown in FIG. 8, and then further modulated in a lookuptable 719 shown in FIG. 9.

Referring to FIG. 9, selectors 713 to 716, 724 to 726, and 731 to 733each receive the output signal from the lookup table 719, and the outputsignals from the lookup tables 747, 748, 758, 767 shown in FIG. 10. Theselectors 713 to 716, 724 to 726, and 731 to 733 also receive givencoefficients, respectively.

An absolutizing circuit 708 absolutizes the output signal from thecoring circuit 601 shown in FIG. 8 and outputs it. Lookup tables 709 and717 modulate the output signal from the absolutizing circuit 708.

A selector 710 selects and outputs, the output signal from the lookuptable 709 and the output signal from the selector 715. A selector 711selects and outputs a given coefficient and a modulation parameter sentfrom the SDRAM 11 through the input DMA channel IN-CH2 or IN-CH3. Aselector 712 selects and outputs the output signal from the selector 716and a modulation parameter sent from the SDRAM 11 through the input DMAchannel IN-CH2 or IN-CH3.

A selector 723 selects and outputs the output signal from the lookuptable 717 and the output signal from the selector 726. A selector 730selects and outputs the output signal from the lookup table 717 and theoutput signal from the selector 733.

Referring to FIG. 8, a multiplier 701 multiplies together the outputsignal from the coring circuit 601 corresponding to Y component(high-frequency component, medium-frequency component, or mixturecomponent thereof) and the output signal from the selector 713 shown inFIG. 9 to provide an output. A multiplier 702 multiplies together theoutput signal from the multiplier 701 and the output signal from theselector 714 shown in FIG. 9 to provide an output. A multiplier 703multiplies together the output signal from the multiplier 702 and theoutput signal from the selector 710 shown in FIG. 9 to provide anoutput. A multiplier 704 multiplies together the output signal from themultiplier 703 and the output signal from the selector 711 shown in FIG.9 to provide an output. The output signal from the multiplier 704 isinputted to the adder 535 through the limiter 534. In this way, theoutput signal from the coring circuit 601 can be arbitrarily modulatedby the multipliers 701 to 704 by using modulating signals generated byarbitrarily modulating pixel unit feature signals, which offers enhancedversatility.

Referring to FIG. 9, a multiplier 720 multiplies together the outputsignal from the selector 724 and the output signal from the coringcircuit 602 corresponding to Cb component, shown in FIG. 8, to providean output. A multiplier 721 multiplies together the output signal fromthe multiplier 720 and the output signal from the selector 725 toprovide an output. A multiplier 722 multiplies together the outputsignal from the multiplier 721 and the output signal from the selector723 to provide an output. A selector 734 selects and outputs the outputsignal from the multiplier 722 and an output signal (maximum value) fromthe noise reduction circuit 507 shown in FIG. 7.

A multiplier 727 multiplies together the output signal from the selector731 and the output signal from the coring circuit 603 corresponding toCr component, shown in FIG. 8, to provide an output. A multiplier 728multiplies together the output signal from the multiplier 727 and theoutput signal from the selector 732 to provide an output. A multiplier729 multiplies together the output signal from the multiplier 728 andthe output signal from the selector 730 to provide an output. A selector735 selects and outputs the output signal from the multiplier 729 and anoutput signal (minimum value) from the noise reduction circuit 507 shownin FIG. 7.

In this way, the color suppression block 26 is capable of arbitrarilymodulating the pixel unit feature signals that are modulating signalsfor modulating signals to be modulated (luminance signal high-frequencycomponent and medium-frequency component, or color signals), by usingthe absolutizing circuits 751, 755, 760, 764, the luminance modulationunits 757, 766, and the lookup tables 758, 767 shown in FIG. 10. Thismakes it possible to arbitrarily modulate the signals to be modulated,according to the purpose, thus offering enhanced versatility.

Second Color-Space Transformation Block 27

The configuration of the second color-space transformation block 27 isillustrated in FIG. 11. As shown in FIG. 11, the second color-spacetransformation block 27 includes a color-space transformation circuit801.

The color-space transformation circuit 801 is capable of performingmatrix operation of 4-channel inputs and 4-channel outputs. For example,it transforms a YCbCr color-space (YUV color-space) image signal into anRGB color-space or YMCW color-space image signal and outputs it. It isalso capable of outputting a YCbCr color-space image signal withouttransforming it, when a coefficient for the color-space transformationcircuit 801 is set to such a value as not to effect color-spacetransformation.

In this way, the spatial filtering block 24, the coring block 25, andthe color suppression block 26 perform various operations using YCbCrcolor-space image signals, and then the second color-spacetransformation block 27 transforms them into, e.g., the RGB color space,whereby an output signal of a desired color space is finally obtainedand the versatility is thus enhanced.

Second Gamma-Correction Block 28

The configuration of the second gamma-correction block 28 is illustratedin FIG. 11. As shown in FIG. 11, the second gamma-correction block 28includes a gamma correction unit 901 that is capable of performingconversion from 12-bit system to 8-bit system using lookup tables etc.,independently for individual color channels C0 to C3. The gammacorrection unit 901 is connected to follow the color-spacetransformation circuit 801.

In this way, the first color-space transformation block 23, the spatialfiltering block 24, the coring block 25, the color suppression block 26,and the second color-space transformation block 27 perform variousoperations using 12-bit-system image signals, and then the secondgamma-correction block 28 performs gamma correction to convert them to8-bit-system image signals. Thus, an output signal of a desired numberof bits is finally obtained and the versatility is thus enhanced.

Resizing Block 29

The configuration of the resizing block 29 is illustrated in FIG. 11.The resizing block 29 is configured to perform image resolutionconversion, and as shown in FIG. 11, the resizing block 29 includes fourresizers 1001 to 1004. The resizer 1001 is capable of processing12-bit-system input signals, and the resizers 1002 to 1004 are capableof processing 8-bit-system input signals. The resizer 1001 is alsocapable of processing 8-bit-system input signals when the high order 4bits are set to “0”.

The image signals that have been gamma-corrected in the gamma correctionunit 901 are inputted to the resizers 1001 to 1004 through selection bya selector 1005. Then, the resizers 1001 to 1004 perform resolutionconversion, and the signals are transferred through the output DMAchannels OUT-CH0 to OUT-CH3 for storing in the SDRAM 11 or display inthe LCD 7. In this way, when the input signal to the resizing block 29is of 8-bit system, not only the resizers 1002 to 1004 but also theresizer 1001 are used to perform resolution conversion. Accordingly, theresizer 1001 is not wasted when the input signal is of 8-bit system.

An image signal that is not subjected to gamma correction in the gammacorrection unit 901 is inputted from the color-space transformationcircuit 801 to the resizer 1001 through the selector 1005. Then, theresolution is converted in the resizer 1001 and it is stored in theSDRAM 11 or recorded in the memory card 13 through the output DMAchannel OUT-CH0.

In this way, while a 12-bit-system signal for main image and an 8-bitsystem signal for display or for thumbnail image are inputted as inputsignals to the resizing block 29, it is possible to select which resizeror resizers are to be used according to the number of bits of the inputsignal, among the resizer 1001 and the resizers 1002 to 1004 configuredto process signals of different numbers of bits. Furthermore, thecircuit scale can be smaller than when all resizers 1001 to 1004 areprovided as resizers capable of processing 12-bit-system input signals.

Two-Dimensional Lookup Table 402

Now, the two-dimensional lookup table 402 shown in FIG. 6 will bedescribed in detail. As explained earlier, the two-dimensional lookuptable 402 is connected to the output terminals of the color channels C1and C2 of the color-space transformation circuit 401, and it transformsthe values of Cb- and Cr-component pixel signals to desired values,using previously set arbitrary transform values representing acorrespondence between pairs of input data values and pairs of outputdata values.

FIG. 12 is a diagram showing an example of setting of transform valuesin the two-dimensional lookup table. The horizontal axis shows thevalues of first data X inputted to the two-dimensional lookup table, andthe vertical axis shows the values of second data Y inputted to thetwo-dimensional lookup table. FIG. 12 shows an example in which both ofthe data X and Y are data with a sign, and the maximum value of bothdata X and Y is “2047” and their minimum value is “−2048”.

In the two-dimensional lookup table, mesh intersections V(p, q) aredefined at intervals of 256 digits both in vertical and horizontaldirections. The values p and q are integers of not less than −8 nor morethan 8. Transform values (x, y), representing output values for data Xand Y, are set at each intersection V(p, q). For example, when the inputvalues to the two-dimensional lookup table are (X, Y)=(256, 512), thedata is inputted to the intersection V(1, 2) and the transform values(x, y)=(256, 512) set at that intersection V(1, 2) are outputted as theoutput values from the two-dimensional lookup table. The input values(X, Y) and the output values (x, y) are equal in this example becauseFIG. 12 shows an example of a non-transforming two-dimensional lookuptable, but arbitrary transform values (x, y) can be set at theintersections V(p, q).

In the two-dimensional lookup table, the transform values (x, y) arediscretely defined only at the mesh intersections V(p, q), and so thecircuit scale can be smaller than when transform values are defined incorrespondence with all input values.

The first color-space transformation block 23 performs color-spacetransformation by real-time processing. Accordingly, when thecolor-space transformation is performed using a three-dimensional lookuptable that involves large amounts of calculations, then the processingin the three-dimensional lookup table will form a bottleneck to delaythe entire processing. Also, a three-dimensional lookup table involves avery large circuit scale. In contrast, using a two-dimensional lookuptable avoids increased circuit scale and achieves high-speed color-spacetransformation without delaying the entire processing.

When the input data has values not defined as an intersection V(p, q),the transform values corresponding to the input values can be calculatedby interpolation using the transform values at the four intersectionssurrounding the input values.

When input values (Xin, Yin) are surrounded by four intersections V₀(X0,Yin0), V₁(X0, Yin1), V₂(X1, Yin0), and V₃(X1, Yin1), then vector dataabout the intersection V₀, (DT0X(X0, Yin0), DT0Y(X0, Yin0)), vector dataabout the intersection V₁, (DT1X(X0, Yin1), DT1Y(X0, Yin1)), vector dataabout the intersection V₂, (DT2X(X1, Yin0), DT2Y(X1, Yin0)), and vectordata about the intersection V₃, (DT3X(X1, Yin1), DT3Y(X1, Yin1)), areobtained. Where X0#Xin<X1 or X0<Xin#X1, Yin0#Yin<Yin1 or Yin0<Yin#Yin1.

Then, the transform values (Xout, Yout) corresponding to the inputvalues (Xin, Yin) are calculated by the interpolation shown below.

$\begin{matrix}{{Xout} = {\left( {{X\; 1} - {X\;{in}}} \right)\text{/}\left( {{X\; 1} - {X\; 0}}\; \right)*\left\{ {\left( {{Y\; 1} - {Y\;{in}}} \right)\text{/}\left( {{Y\; 1} - {Y0}} \right)*{{{DT}0X}\left( {{X0},} \right.}} \right.}} \\{\left. {\left. {Yin0} \right) + {\left( {{Yin} - {Y\; 0}} \right)\text{/}\left( {{Y\; 1} - {Y\; 0}} \right)*{DT}\; 1\; X\;\left( {{X\; 0},{Y\;{in}\; 1}} \right)}} \right\} +} \\{\left( {{X\;{in}} - {X\; 0}} \right)\text{/}\left( {{X\; 1} - {X\; 0}} \right)*\left\{ {\left( {{Y\; 1} - {Y\;{in}}} \right)\text{/}\left( {{Y\; 1} - {Y\; 0}} \right)*{DT}\; 2\; X}\; \right.} \\\left. {\left( {{X\; 0},{Y\;{in}\mspace{11mu} 0}} \right) + {\left( {{Y\;{in}} - {Y\; 0}} \right)*{DT}\; 3{X\left( {{X\; 0},{Y\;{in}\; 1}} \right)}}} \right\}\end{matrix}$ $\begin{matrix}{{Y{out}} = {\left( {{X\; 1} - {X\;{in}}} \right)\text{/}\left( {{X\mspace{11mu} 1} - {X\mspace{11mu} 0}} \right)*\left\{ {\left( {{Y\; 1} - {Y\;{in}}} \right)\text{/}\left( {{Y\; 1} - {Y\; 0}} \right)*} \right.}} \\{\left. {\left. {{{DT}\; 0\; Y\;\left( {{X\; 0},{Y\;{in}\; 0}} \right)} + {\left( {{Y\;{in}} - {Y\; 0}} \right)\text{/}Y\; 1} - {Y\; 0}} \right)*{DT}\; 1\;{Y\left( {{X\; 0},{Y\;{in}\mspace{11mu} 1}} \right)}} \right\} +} \\{\left( {{X\;{in}} - {X\; 0}} \right)\left( {{X\; 1} - {X\; 0}} \right)*\left\{ {\left( {{Y\mspace{11mu} 1} - {Y\;{in}}} \right)\text{/}\left( {{Y\; 1}\; - {Y\; 0}} \right)*} \right.} \\\left. {{{DT}\mspace{11mu} 2\mspace{11mu}{Y\left( {{X\; 1},{Y\;{in}\mspace{11mu} 0}} \right)}} + {\left( {{Y\;{in}} - {Y\; 0}} \right)\text{/}\left( {{Y\; 1} - {Y\; 0}} \right)*{DT}\; 3{Y\left( {{X\; 1},{Y\;{in}\mspace{11mu} 1}} \right)}}} \right\}\end{matrix}$

In the two-dimensional lookup table 402 shown in FIG. 6, one of thevertical and horizontal axes shows Cb-component values and the othershows Cr-component values. Then, arbitrary transform values (Cb, Cr) areset at individual intersections V(p, q) using the values of Cb componentand Cr component, whereby the two-dimensional lookup table 402 appliesarbitrary color transformation to the image signal outputted from thecolor-space transformation circuit 401. For example, it applies colortransformation for enhancing the saturation of skin color of people orblue of the sky, or color transformation for matching a particular colorshifted from a standard color chart with a color on the color chart.

Since the YCbCr color space only involves two-channels of color signals,it is possible to transform all two-channel color signals outputted fromthe color-space transformation circuit 401 by using the two-dimensionallookup table 402, without a need to use a three-dimensional lookuptable.

FIG. 13 is a diagram showing another example of the two-dimensionallookup table. While the intersections V(p, q) of FIG. 12 are arranged atequal intervals, the mesh of the example of FIG. 13 is finer in acentral area of the two-dimensional lookup table, and thus a largernumber of intersections are defined in the central area. That is, thedegree of discreteness of transform values is set smaller in the area inwhich the absolute values of input data are smaller (region near gray)than in the area where the absolute values are larger. In this way, thetransform values are densely set in the region near gray, which makes itpossible to apply finer transformation in the area near gray that ismore frequently contained in normal picture images.

FIGS. 14 and 15 are circuit diagrams showing a first example of specificconfiguration of the two-dimensional lookup table. FIGS. 14 and 15 areconnected together at the connection points N₁₀₁ to N₁₁₁ of the samenumbers. Also, FIGS. 16 and 17 are circuit diagrams showing a secondexample of specific configuration of the two-dimensional lookup table.FIGS. 16 and 17 are connected together at the connection points N₂₀₁ toN₂₁₀ of the same numbers. The first example shows a configuration usingsingle-port memory, and the second example shows a configuration using4-port memory. The first example requires four clocks to input fourpairs of transform values for interpolation into the memory, while thesecond example requires only a single clock because it cansimultaneously input four pairs of transform values into the memory.

While the invention has been described in detail, the foregoingdescription is in all aspects illustrative and not restrictive. It isunderstood that numerous other modifications and variations can bedevised without departing from the scope of the invention.

What is claimed is:
 1. An image processing apparatus comprising: aplurality of image processing blocks that are capable of real-timeprocessing of an image signal inputted from outside, said plurality ofimage processing blocks including, a spatial filtering block thatapplies a spatial filtering to an input signal; and a signal modulationblock that applies a modulation to an output signal from said spatialfiltering block as a signal to be modulated, said spatial filteringblock comprising a spatial filter that outputs a luminance signal thathas been subjected to said spatial filtering, and said signal modulationblock comprising, a plurality of low-pass filters each corresponding toa channel, the plurality of low-pass filters applying a low-passfiltering to a pixel unit feature signal corresponding to one or morechannels; and an operator that modulates said luminance signal by usingsaid pixel unit feature signal that has been subjected to said low-passfiltering, said plurality of image processing blocks further including apixel interpolation block that applies a pixel interpolation to an inputsignal, and said pixel unit feature signal including at least one of acolor level signal, a Sobel filter output signal, and a correlationsignal indicating a correlation with surrounding pixels, each of saidcolor level signal, said Sobel filter output signal, and saidcorrelation signal being generated in said pixel interpolation block. 2.An image processing apparatus comprising: a plurality of imageprocessing blocks that are capable of real-time processing of an imagesignal inputted from outside, said plurality of image processing blocksincluding, a spatial filtering block that applies a spatial filtering toan input signal; and a signal modulation block that applies a modulationto an output signal from said spatial filtering block as a signal to bemodulated, said spatial filtering block comprising a spatial filter thatoutputs a color signal that has been subjected to said spatialfiltering, and said signal modulation block comprising, a plurality oflow-pass filters each corresponding to a channel, the plurality oflow-pass filters applying a low-pass filtering to a pixel unit featuresignal corresponding to one or more channels; and an operator thatmodulates said color signal by using said pixel unit feature signal thathas been subjected to said low-pass filtering, said plurality of imageprocessing blocks further including a pixel interpolation block thatapplies a pixel interpolation to an input signal, and said pixel unitfeature signal including at least one of a color level signal, a Sobelfilter output signal, and a correlation signal indicating a correlationwith surrounding pixels, each of said color level signal, said Sobelfilter output signal, and said correlation signal being generated insaid pixel interpolation block.
 3. The image processing apparatusaccording to claim 1, wherein said signal modulation block modulatessaid signal to be modulated, with a signal obtained by applying to saidpixel unit feature signal, in addition to said low-pass filtering, atleast one of: absolutization; modulation using a luminance level of acenter pixel signal as an original pixel signal; modulation using aluminance level of an output signal of a low-pass filter included insaid spatial filtering block; and modulation using a lookup table. 4.The image processing apparatus according to claim 2, wherein said signalmodulation block modulates said signal to be modulated, with a signalobtained by applying to said pixel unit feature signal, in addition tosaid low-pass filtering, at least one of: absolutization; modulationusing a luminance level of a center pixel signal as an original pixelsignal; modulation using a luminance level of an output signal of alow-pass filter included in said spatial filtering block; and modulationusing a lookup table.
 5. The image processing apparatus according toclaim 1, wherein each low-pass filter corresponds to a differentchannel.
 6. The image processing apparatus according to claim 2, whereineach low-pass filter corresponds to a different channel.
 7. An imageprocessing apparatus comprising: a plurality of image processing blocks,implemented by processing circuitry, that are capable of real-timeprocessing of an image signal inputted from outside, said plurality ofimage processing blocks including, a spatial filtering block thatapplies a spatial filtering to an input signal; and a signal modulationblock that applies a modulation to an output signal from said spatialfiltering block as a signal to be modulated, said spatial filteringblock comprising a spatial filter that outputs a luminance signal orcolor signal that has been subjected to said spatial filtering, and saidsignal modulation block comprising, a plurality of low-pass filters eachcorresponding to a channel, the plurality of low-pass filters applying alow-pass filtering to a pixel unit feature signal corresponding to oneor more channels; and an operator that modulates said luminance signalor color signal by using said pixel unit feature signal that has beensubjected to said low-pass filtering, said plurality of image processingblocks further including a pixel interpolation block that applies apixel interpolation to an input signal, and said pixel unit featuresignal including at least one of a color level signal, a Sobel filteroutput signal, and a correlation signal indicating a correlation withsurrounding pixels, each of said color level signal, said Sobel filteroutput signal, and said correlation signal being generated in said pixelinterpolation block.